CLAIMS OF THE INVENTION 



We claim: 

5 LA method for two-dimensional or three-dimensional spatial manipulation 

or two-dimensional or three-dimensional entertainment, comprising: 

generating one or more interface devices to alter and generate one or more two- 
dimensional or three-dimensional virtual objects, wherein said devices can control N 
degrees of freedom of said virtual objects; 
10 associating said interface devices in conjunction with each other to alter one or 

more two-dimensional or three-dimensional virtual components; 

providing one or more three-dimensional virtual tools to a user for said spatial 
manipulation or said two-dimensional or said three-dimensional entertainment; and 
providing a plurality of video game controllers to said user for said spatial 
15 manipulation or said two-dimensional or said three-dimensional entertainment. 

2. The method of claim 1 wherein said interface devices are digital input 
devices. 

20 3. The method of claim 1 wherein said interface devices are physical input 

devices. 

4. The method of claim 1 wherein said virtual components are a software 
representation of one or more physical input devices, wherein said software 
25 representation has a two-dimensional or three-dimensional rendering associated with it. 
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5. The method of claim 1 wherein one of said virtual components is a two- 
dimensional GUI. 

6. The method of claim 1 wherein one of said virtual component is a three- 
5 dimensional GUI. 

7. The method of claim 1 wherein one of said virtual components is a cursor 
on a computer screen. 

10 8. The method of claim 1 wherein one of said interface devices is a grabbing 

tool. 

9. The method of claim 8 wherein said grabbing tool has a physical form 
resembling a pair of kitchen tongs. 

15 

10. The method of claim 8 wherein said grabbing tool has a physical form 
resembling a pair of pincers. 

1 1 . The method of claim 8 wherein said grabbing tool has a physical form 
20 resembling a pair of scissors. 

12. The method of claim 8 wherein said grabbing tool has a physical form 
resembling a pair of tweezers. 

25 13. The method of claim 8 wherein a first virtual form of said grabbing tool is 

an iconic virtual component only. 
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14. The method of claim 8 wherein a second virtual form of said grabbing tool 
is a first iconic virtual component coupled with a second virtual component that coincides 
with said tool's physical form. 

5 15. The method of claim 8 wherein a third virtual form of said grabbing tool is 

a virtual component coinciding with said tool's physical form. 

16. The method of claim 8 wherein a fourth virtual form of said grabbing tool 
is a lack of virtual depiction. 

10 

17. The method of claim 8 further comprises: 

altering the relationship between said grabbing tool and its corresponding three- 
dimensional virtual component; 

mapping said grabbing tool to said corresponding virtual component; 
15 controlling position of said corresponding virtual component to one of said virtual 
objects; and 

generating an iconic form when said corresponding virtual component is close enough to 
react with one of said virtual objects. 

20 18. The method of claim 1 7 wherein said controlling comprises embedding 

one or more sensors within said virtual component. 

19. The method of claim 18 wherein said sensors is any one of magnetic, 
optical, or inertial sensors. 

25 

20. The method of claim 17 wherein said controlling position comprises 
integrating said virtual component within a controlling environment. 
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21 . The method of claim 20 wherein said controlling environment is a camera. 



22. The method of claim 8 wherein said grabbing tool has a plurality of 
controls to activate one or more functions. 

5 

23. The method of claim 22 wherein said plurality of controls comprises 
buttons, joysticks, scroll wheels, or foot pedals embedded in said virtual component. 

24. The method of claim 22 wherein one of said functions is to display to said 
10 user a virtual menu consisting of one or more choices for said user to choose from. 

25. The method of claim 22 wherein one of said functions is to toggle between 
a first action mode and a second action mode. 

15 26. The method of claim 25 wherein said first action mode is a default action 

mode. 

27. The method of claim 22 wherein one of said functions is to release a 
plurality of virtual weapons of one or more type. 

20 

28. The method of claim 1 wherein one of said interface devices is a pointing 

tool. 

29. The method of claim 28 wherein said pointing tool has a physical form 
25 resembling a firearm. 

30. The method of claim 29 wherein said firearm is a gun. 
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31 . The method of claim 28 wherein said pointing tool has a physical form 
resembling a laser pointer. 

32. The method of claim 28 wherein said pointing tool has a physical form 
5 resembling a camera. 

33. The method of claim 28 wherein said pointing tool has a physical form 
resembling a pointing hand. 

10 34. The method of claim 28 wherein said pointing tool has a physical form 

resembling a stick. 

35. The method of claim 28 wherein said pointing tool has a physical form 
resembling a flashlight. 

15 

36. The method of claim 28 wherein said pointing tool has a physical form 
resembling a spray-paint can. 

37. The method of claim 28 wherein said pointing tool has a physical form 
20 resembling a glue-gun. 

38. The method of claim 28 wherein a first virtual form of said pointing tool is 
an iconic virtual component only. 

25 39. The method of claim 28 wherein a second virtual form of said pointing 

tool is a first iconic virtual component coupled to a second virtual component that 
coincides with said tool's physical form. 
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40. The method of claim 28 wherein a third virtual form of said pointing tool 
is a virtual component coinciding with said tool's physical form. 

41 . The method of claim 28 wherein a fourth virtual form of said pointing tool 
5 is a lack of virtual depiction. 

42. The method of claim 28 further comprises: 

altering the relationship between said pointing tool and its corresponding three- 
dimensional virtual component; 
10 mapping said pointing tool to said corresponding virtual component; 

controlling position of said corresponding virtual component to one of said virtual 
objects; and 

generating an iconic form when said corresponding virtual component is close enough to 
react with one of said virtual objects. 

15 

43. The method of claim 42 wherein said controlling comprises embedding 
one or more sensors within said virtual component. 

44. The method of claim 43 wherein said sensors is any one of magnetic, 
20 optical, or inertial sensors. 

45. The method of claim 42 wherein said controlling position comprises 
integrating said virtual component within a controlling environment. 

25 46. The method of claim 45 wherein said controlling environment is a camera. 
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47. The method of claim 28 wherein said pointing tool has a plurality of 
controls to activate one or more functions. 

48. The method of claim 47 wherein said plurality of controls comprises 
5 buttons, joysticks, scroll wheels, or foot pedals embedded in said virtual component. 

49. The method of claim 47 wherein one of said functions is to display to said 
user a virtual menu consisting of one or more choices for said user to choose from. 

10 50. The method of claim 47 wherein one of said functions is to toggle between 

a first action mode and a second action mode. 

51 . The method of claim 50 wherein said first action mode is a default action 

mode. 

15 

52. The method of claim 47 wherein one of said functions is to release a 
plurality of virtual weapons of one or more type. 

53. The method of claim 1 wherein one of said interface devices is a gripping 

20 tool. 

54. The method of claim 53 wherein said gripping tool has a physical form 
resembling a handle. 

25 55. The method of claim 54 wherein said handle is a sword handle. 

56. The method of claim 54 wherein said handle is a shovel handle. 
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57. The method of claim 53 wherein a first function of said gripping tool is to 
place said virtual objects in a three-dimensional space. 



58. The method of claim 53 wherein a second function of said gripping tool is 
5 to draw one or more paths between two or more of said virtual objects. 

59. The method of claim 53 wherein a first virtual form of said gripping tool is 
an iconic virtual component only. 

10 60. The method of claim 53 wherein a second virtual form of said gripping 

tool is a first iconic virtual component coupled with a second virtual component that 
coincides with said tool's physical form. 

61 . The method of claim 53 wherein a third virtual form of said gripping tool 
15 is a virtual component coinciding with said tool's physical form. 

62. The method of claim 53 wherein a fourth virtual form of said gripping tool 
is a lack of virtual depiction. 

20 63. The method of claim 53 further comprises: 

altering the relationship between said gripping tool and its corresponding three- 
dimensional virtual component; 

mapping said gripping tool to said corresponding virtual component; 
controlling position of said corresponding virtual component to one of said virtual 
25 objects; and 

generating an iconic form when said corresponding virtual component is close enough to 
react with one of said virtual objects. 
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64. The method of claim 63 wherein said controlling comprises embedding 
one or more sensors within said virtual component. 

65. The method of claim 64 wherein said sensors is any one of magnetic, 
5 optical, or inertial sensors. 

66. The method of claim 63 wherein said controlling position comprises 
integrating said virtual component within a controlling environment. 

10 67. The method of claim 66 wherein said controlling environment is a camera. 

68. The method of claim 53 wherein said grabbing tool has a plurality of 
controls to activate one or more functions. 

15 69. The method of claim 68 wherein said plurality of controls comprises 

buttons, joysticks, scroll wheels, or foot pedals embedded in said virtual component. 

70. The method of claim 68 wherein one of said functions is to display to said 
user a virtual menu consisting of one or more choices for said user to choose from. 

20 

71 . The method of claim 68 wherein one of said functions is to toggle between 
a first action mode and a second action mode. 

72. The method of claim 71 wherein said first action mode is a default action 

25 mode. 
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73. The method of claim 68 wherein one of said functions is to release a 
plurality of virtual weapons of one or more type. 

74. A method to draw a virtual object in a two-dimensional or three- 
5 dimensional space or in a two-dimensional or three-dimensional entertainment 

environment, comprising: 

using two or more physical input devices coincidentally. 

75. The method of claim 74 wherein using two or more physical input devices 
10 further comprises: 

using said grabbing tool in combination with said pointing tool to create a solid 
curve or volume. 

76. The method of claim 75 wherein said using grabbing tool in combination 
15 with said pointing tool further comprises: 

using said grabbing tool to grab said pointing tool's virtual object; 
bending said virtual object of said pointing tool with said grabbing tool; and 
sweeping said pointing tool in said three-dimensional space or entertainment 
environment to create said curve or volume. 

20 

77. The method of claim 76 wherein said sweeping causes a three-dimensional 
solid curve if said curve is not a closed loop. 

78. The method of claim 76 wherein said sweeping causes a three-dimensional 
25 solid volume if said curve is a closed loop. 
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79. A method to assemble and rearrange a virtual molecule in a two- 
dimensional or three-dimensional space or in a two-dimensional or three-dimensional 
entertainment environment, comprising: 

using said grabbing tool in combination with said gripping tool and said pointing 

tool. 

80. The method of claim 79 wherein said using further comprises: 

using one of said gripping tool's plurality of controls to activate one or more of 
said gripping tool's functions to construct said molecule; 

using said pointing tool to draw one or more bonds of said molecule; 

using said grabbing tool to move said molecule to a position in said three- 
dimensional space or entertainment environment for easier drawing of said bonds; and 

using said gripping tool to break any one or more of said bonds. 

81. A method to change placement of a virtual object in a two-dimensional or 
three-dimensional space or in a two-dimensional or three-dimensional entertainment 
environment, comprising: 

using two or more of said physical input devices coincidentally. 

82. The method of claim 81 wherein said using two or more physical input 
devices further comprises: 

using a first grabbing tool in conjunction with a second grabbing tool to rotate 
said virtual object in two-dimensional or three-dimensional space or in a three- 
dimensional entertainment environment. 

83. The method of claim 82 wherein said using further comprises: 
grabbing a first extremity of said virtual object with said first grabbing tool; 
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grabbing a second extremity of said virtual object with said second grabbing tool; 

and 

rotating said virtual object to a desired position in said two-dimensional or three- 
dimensional space or three-dimensional entertainment environment by moving one or 
both of said first grabbing tool and said second grabbing tool. 

84. A method to deform a virtual object in a two-dimensional or three- 
dimensional space or in a two-dimensional or three-dimensional entertainment 
environment, comprising: 

using two or more of said physical input devices coincidentally. 

85. The method of claim 84 wherein said using two or more physical input 
devices further comprises: 

using a first grabbing tool in conjunction with a second grabbing tool to stretch 
said virtual object in two-dimensional or three-dimensional space or in a three- 
dimensional entertainment environment. 

86. The method of claim 84 wherein said using two or more physical input 
devices further comprises: 

using a first grabbing tool in conjunction with a second grabbing tool to twist said 
virtual object in two-dimensional or three-dimensional space or in a three-dimensional 
entertainment environment. 

87. A method to alter a physical input device's virtual object in a two- 
dimensional or three-dimensional space or in a two-dimensional or three-dimensional 
entertainment environment, comprising: 

using two or more of said physical input devices coincidentally. 
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88. The method of claim 87 wherein said using two or more physical input 
devices further comprises: 

using a first grabbing tool to modify an axis of rotation of a second grabbing tool 
in two-dimensional or three-dimensional space or in a three-dimensional entertainment 
environment. 

89. The method of claim 88 wherein said using further comprises: 
using said first grabbing tool to grab said second grabbing tool's virtual 

component; 

using said first grabbing tool to move said virtual component of said second 
grabbing tool to a desired location in two-dimensional or three-dimensional space or 
three-dimensional entertainment environment in relationship to said second grabbing 
tool; and 

using said second grabbing tool to rotate said virtual object once said virtual 
component is positioned in said desired location. 

90. A method to specify a point in a two-dimensional or three-dimensional 
space or in a two-dimensional or three-dimensional entertainment environment, 
comprising: 

using two or more of said physical input devices coincidentally. 

91 . The method of claim 90 wherein said using two or more physical input 
devices further comprises: 

using a first pointing tool and a second grabbing tool to specify a point in two- 
dimensional or three-dimensional space or in a three-dimensional entertainment 
environment. 
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92. The method of claim 91 wherein said using further comprises: 
intersecting a virtual object of said first pointing tool and a virtual object of said 

second pointing tool to denote said point in said two-dimensional or three-dimensional 
space or in a three-dimensional entertainment. 

93. The method of claim 92 wherein said virtual object of said first pointing 
tool and said second pointing tool resembles a laser beam. 

94. The method of claim 92 wherein said virtual object of said second 
pointing tool resembles a plane emanating from a barrel of a gun. 

95. A method for altering the spatial relationship between a physical input 
device and one or more of its two or three-dimensional virtual components, comprising: 

using a first physical input device and a second physical input device 
coincidentally. 

96. The method of claim 95 wherein said using a first and a second physical 
input device further comprises: 

using said first physical input device and said second physical input device to cut 
a virtual object located at a position in said two-dimensional or three-dimensional space 
or three-dimensional entertainment environment. 

97. The method of claim 96 wherein said using further comprises: 

using said first physical input device to grab said second physical input device's 
virtual object; 
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using said first physical device to lengthen said virtual object until it reaches a 
desired length; and 

using said second physical input device to cut said virtual object. 

98. A method to map a plurality of virtual components to one physical input 
device, comprising: 

using a virtual menu to map said plurality of virtual components to said physical 
input device. 

99. The method of claim 98 wherein said virtual menu is activated via an 
additional control on said physical input device. 

100. A method to change a plurality of virtual components mapped to a 
physical input device, comprising: 

using a virtual menu to change said plurality of virtual components mapped to 
said physical device. 

101 . The method of claim 100 wherein said virtual menu is activated via an 
additional control on said physical input device. 

102. The method of claim 1 wherein each of said three-dimensional virtual 
tools has a virtual form further comprising: 

an iconic virtual component only; 

an iconic virtual component along with a virtual component that resembles said 
tool's physical form; 

a virtual component only that resembles said tool's physical form; and 

a virtual component lacking virtual depiction. 
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103. The method of claim 102 further comprises: 
controlling position of three-dimensional virtual component to one of said virtual objects. 

5 104. The method of claim 103 wherein said controlling comprises embedding 

one or more sensors within said virtual component. 

105. The method of claim 104 wherein said sensors is any one of magnetic, 
optical, or inertial sensors. 

10 

106. The method of claim 104 wherein said sensors can activate one or more 
functions. 

107. The method of claim 106 wherein said functions further comprises: 
15 monitoring when said virtual tool is moved from a default location; and 

monitoring when said virtual tool is being held by said user. 

108. The method of claim 103 wherein said controlling comprises integrating 
said virtual component within a controlling environment. 

20 

109. The method of claim 108 wherein said controlling environment is a 

camera. 

110. The method of claim 1 wherein one of said three-dimensional virtual tool 
25 is an eraser tool, wherein said tool is used to remove a region of a virtual surface. 
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111. The method of claim 1 wherein one of said three-dimensional virtual tool 
is a deformation tool, wherein said tool is used to deform the geometry of said virtual 
component. 

5 112. The method of claim 1 wherein one of said three-dimensional virtual tool 

is a smoothing tool, wherein said tool is used to smooth a surface of said virtual 
component. 

113. The method of claim 1 wherein one of said three-dimensional virtual tool 
10 is a spray-painting tool, wherein said tool is used to spray said virtual component with 

virtual paint. 

1 14. The method of claim 1 wherein one of said three-dimensional virtual tool 
is a texture creation tool, wherein said tool is used to spray a texture on said virtual 

15 component. 

115. The method of claim 110 wherein said eraser tool has a plurality of 
controls to activate one or more functions. 

20 116. The method of claim 1 15 wherein said plurality of controls comprises 

buttons, joysticks, scroll wheels, or foot pedals embedded in said tool. 

1 17. The method of claim 115 wherein one of said functions is to change a size 
of a default erasing region. 

25 

118. The method of claim 1 1 5 wherein one of said functions is to display to 
said user a virtual menu consisting of one or more choices for said user to choose from. 
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119. The method of claim 115 wherein one of said functions is to toggle 
between a first action mode and a second action mode. 

120. The method of claim 1 19 wherein said first action mode is a default action 

mode. 

121 . The method of claim 111 wherein said deformation tool has a plurality of 
controls to activate one or more functions. 

122. The method of claim 121 wherein said plurality of controls comprises 
buttons, joysticks, scroll wheels, or foot pedals embedded in said tool. 

123. The method of claim 121 wherein one of said functions is to change a 
sensitivity of deformation of said virtual component. 

124. The method of claim 121 wherein one of said functions is to display to 
said user a virtual menu consisting of one or more choices for said user to choose from. 

125. The method of claim 121 wherein one of said functions is to toggle 
between a first action mode and a second action mode. 

126. The method of claim 125 wherein said first action mode is a default action 

mode. 

127. The method of claim 1 12 wherein said smoothing tool has a plurality of 
controls to activate one or more functions. 
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128. The method of claim 127 wherein said plurality of controls comprises 
buttons, joysticks, scroll wheels, or foot pedals embedded in said tool. 

5 129. The method of claim 127 wherein one of said functions is to change a size 

of a soothing region of said virtual component. 

130. The method of claim 127 wherein one of said functions is to change a 
degree of smoothing of said virtual component. 

10 

131. The method of claim 1 27 wherein one of said functions is to display to 
said user a virtual menu consisting of one or more choices for said user to choose from. 

132. The method of claim 127 wherein one of said functions is to toggle 
15 between a first action mode and a second action mode. 

133. The method of claim 132 wherein said first action mode is a default action 

mode. 

20 134. The method of claim 113 wherein said spray-painting tool has a plurality 

of controls to activate one or more functions. 

135. The method of claim 134 wherein said plurality of controls comprises 
buttons, joysticks, scroll wheels, or foot pedals embedded in said tool. 

25 

136. The method of claim 134 wherein one of said functions is to change a 
color of a paint sprayed on said virtual component. 
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137. The method of claim 134 wherein one of said functions is to change a flow 
rate of said paint being sprayed on said virtual component. 

5 138. The method of claim 134 wherein one of said functions is to display to 

said user a virtual menu consisting of one or more choices for said user to choose from. 

139. The method of claim 134 wherein one of said functions is to toggle 
between a first action mode and a second action mode. 

10 

140. The method of claim 139 wherein said first action mode is a default action 

mode. 

141 . The method of claim 1 14 wherein said texture creation tool has a plurality 
15 of controls to activate one or more functions. 

142. The method of claim 141 wherein said plurality of controls comprises 
buttons, joysticks, scroll wheels, or foot pedals embedded in said tool. 

20 143. The method of claim 141 wherein one of said functions is to change a type 

of texture sprayed on said virtual component. 

144. The method of claim 141 wherein one of said functions is to change a flow 
rate of said texture being sprayed on said virtual component. 

25 

145. The method of claim 141 wherein one of said functions is to display to 
said user a virtual menu consisting of one or more choices for said user to choose from. 
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146. The method of claim 141 wherein one of said functions is to toggle 
between a first action mode and a second action mode. 

147. The method of claim 146 wherein said first action mode is a default action 

mode. 

148. The method of claim 1 wherein one of said controller is a grabbing 
controller, wherein said controller is used to grab said one or more virtual objects in said 
entertainment environment. 

149. The method of claim 148 wherein said grabbing controller is used to rotate 
said one or more virtual objects. 

150. The method of claim 148 wherein said grabbing controller is used to move 
said one or more virtual objects. 

151 . The method of claim 1 wherein one of said controllers is a slicing 
controller, wherein said controller is used to slice and relocate said one or more virtual 
objects in said entertainment environment. 

152. The method of claim 151 wherein one of said slicing controller's physical 
shape is a handle and one of its virtual component is a laser beam. 

153. The method of claim 152 wherein said handle shaped slicing controller is 
used to drop objects in said entertainment environment. 
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1 54. The method of claim 1 wherein one of said controllers is a pointing 
controller, wherein said controller is used to shoot said one or more virtual objects in said 
entertainment environment. 

5 155. The method of claim 154 wherein said pointing controller is used to select 

said one or more virtual objects in said entertainment environment. 

156. The method of claim 154 wherein said pointing controller is used to grab 
or rearrange said one or more virtual objects in said entertainment environment. 

10 

157. The method of claim 1 wherein one of said controllers is a drawing 
controller, wherein said controller is used to draw a stroke in said entertainment 
environment. 

15 158. The method of claim 1 57 wherein said stroke is drawn freehand by said 

user in said entertainment environment. 

1 59. The method of claim 1 57 wherein said stroke is drawn using said handle 
by said user in said entertainment environment. 

20 

160. The method of claim 1 wherein one of said controllers is a navigation 
controller, wherein said controller is used to navigate said user in said entertainment 
environment. 

25 161 . The method of claim 1 wherein N degrees of freedom is 6 degrees of 

freedom. 
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162. The method of claim 1 wherein N degrees of freedom is more than 6 
degrees of freedom. 
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